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ABSTRACT 

An evaluation of the usability, effectiveness, and 
acceptance in a job environment vas performed on a general purpose . 
simulator using a simulation of a radar system* General purpose 
simulators permit sharing of a programable capacity among 
simulations, thus providing economical hands-on training and training 
not asually economically available by other means. Training and 
exercises in malfunction isolation vere given Air National Guard 
personnel. Data obtained using questionnaires, a performance test, 
and interviews indicated that the simulation vas usable, effective, 
and acceptable. Detailed information concerning: (1) the experience 
o£ the personnel receiving training, (2) learning and learning 
potential, (3) an evaluation of training potential by qualified 
personnel, and (4) attitudes and acceptance by field personnel vas 
obtained and is reported. The goals of the training program vere 
achieved to a satisfactory level. Actual performance scores also 
indicated training goal achievement. The pre- and posttest 
questionnaires are appended. (Author/BP) 
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TRYOUT OF A GENERAL PURPOSE SIMULATOR IN 
AN AIR NATIONAL GUARD TRAINING ENVIRONMENT 



I. irfTRODUCnON 

The development of economical and effective simulation is necessary to provide student active 
practice of the task being learned. One possible technique for providing low cost hands-on training is 
general purpose simulation. Previous research under an Air Force contract has indicated that general 
purpose simulation in formal technical training may be feasible and cost-effective in many applications. 
This report evaluates the use of general purpose simulation in an on-the-job-training(OJT) environment. 

General purpose simulation includes a family of simulators sharing certain features. Specific training 
requirements are met by modularized configurations which are seated upon a standard mainframe and 
control system. The mainframe console consists of a programmable digital computer, a random access 
rear-screen projection system, a computer controlled meter movement, and a control center (a digital 
keyboard with digital display indicators). The modularized configuring capability is derived from a tusk 
specific simulation display panel (a pictorial/schematic model of selected equipment with appropriate 
control circuitry), magnetic tape cassette computer program, a plastic embedded slide projection disc (for 
the custjmized projector), and interchangeable meterfaces and probes (when included), A specific 
simulator is activated by installing the appropriate slide projection disc, and the desired simulation display 
panel, on the mainframe and loading the matched magnetic tape cassette into the dedicated computer. The 
mainframe console holds the simulation display panel which, when locked in place, is controlled by the 
computer appropriately programmed. The control console keyboard then is used to select the conditions of 
simulator operation (normal or various preselected malfunction modes). 

General purpose simulators (GPS) permit sharing of programmable control equipment in a variety of 
specific applications. A simulation representing a radar system can rapidly be reconfigured to represent a 
camera, a washing machine, a fire control sysiem, or the ignition sysiem of an engine. When a simulator 
configuration is installed normal system operation is simulated unless prespccified malfunctions are entered 
by means of the control panel. During malfunction operation, correct isolation of the malfunction 
automatically returns the operation to normal mode. The digital display indicators show elapsed time and 
the number of replacements. The number of systems tests can also be recorded when appropriate. 

An evaluation of general purpose simulation, to inciease hands-on capabilities in Air Force technical 
training in a resident environment, was conducted and results arc documented in a technical report which is 
in press. A GPS marketed as the EC-II was procured from the Educational Computer Corporation. 
Following special factory training and an analysis of the course learning requirements, a faceplate 
(simulation display panel) and a sequence of slides were designed so as to simulate the APQ 126 Radar 
System v/hich is associated with the A7D aircraft. The computer program, implementing the faceplate and 
selected slides, was developed by Educational Computer Corporation and procured from them. The finished 
product is pictured in Figure I. 

Evaluation of the resulting simulatoi was performed in several ways. Cost comparisons, effectiveness 
in learning, learner attitudes, and an analysis of design difficulty were performed m a selected Avionics 
course. Results showed that the initial cost for an operational GPS was less than ten percent that of the 
equivalent actual equipment trainer (AET). Moreover, training in techniques of malfunction isolation was 
not possible using the comparable AET. The GPS provided significant learning opportunities not previously 
available, nor readily feasible using other modes. 

Training on both the GPS and the AET pemiitted student learning to the criterion performance level 
on normal operational procedures. No interference occurred when transferring cither from the GPS to the 
AET or the AET lo the GPS. As stated above, practice and feedback on the isolation of malfunctions was 
provided only on the GPS; a capability previously unavailable to the course of instruction. 

A Held evaluation using Held personnel for instructors was requested by the Air National Guard lo 
determine if general purpose simulation would assist in their OJT program. While the GPS was shown 
cost cffecrivc and feasible in a technical training resident school environment, its use in job training un the 
llighl line hi»d not been evaluated. The purpose of the present study was to evaluate the usability, 
effectiveness, and acccpiability (by instructors ;md students) of gc.ieral purpose simulation. The simulation 
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five JoT'^porsr^^^^^^ iTS^r;;^ '7' ^" -P-^-ced 

program on the GPS by an experienced instructor 1 a " r^" ''^^ '26 simulation 

L^ter a threeJevel mechanic vS^preWo: e 'erie^^^^^ Lowry AFB, Colorado, 

as an instructor. ^ expenence m teachmg high school was trained so as to perfomi 

Buckle^Si'Sor^^^^^^^^^ w^Pons control shop mechanics to the GPS at 

giving him practice on isolating sorpSmined ^^^^^^ f'* T '° °P"^'« simulator and 

in malfunction isolation. A q^e^trnaire (tlpS^^^^^ of the mechanics was tested 

simulator. This questionnaire was usefto de e'mine pr^^^^^^ ^"'"^'"e the 
of training potential, attitudes, and accept^ncT Ae tLe T 

isolation also were recorded by each techSan slTce ulh ' '^P'^"'"""'" for malfunction 
indication of acceptability and^e ed Su^e s ^ 

Simulation approach to traLng we^reSown totc'Srha!?: bT^^^^^^^ ^ 

.echni^r p'Se" ^7 i^th^L^TerM^^^^^^^^ ^"^T' ^« ''^-•leve, 

hour and one^ialf instruction was pro "deTto ch^ ^^^'^"^ °P"^»i«"- A" additional 

in malfunction isolation. QuestionE we e aTo^^iM h r "7 ^'"''«"' '"'ed 

experience,Iearning, attitudes and aTceptaTce ^^^^'^ ^"'l 'fining to determine previous 

Repair^Sh^S^w^rS.^^^^^^^^^ the Avoinics Aerospace Ground Equipment 

manner as previously described for the ie^e p^Ln^S^ra^iig '''' 

Person^in^ personnel. ^1 assigned 



in. RESULTS 

^^^J^^^'^^Z ^:srel^;t.tVng1^^^^^^^ experience and 

expenenced personnel, and (4) atlitSdinal and aSpta^^^^^^^^^^ °' '^^'"'"6 Potential by 

hadb^reSa?ei;Vy\\^^^^^^^^ 

m years. All were familiar withT 126 I^^dt^^^^^ 

troubleshoot this system. ^ '^^ ^y'^'^'" reported themselves qualified to 

radar system. It was reported to the experimeLrTh.VS!? '7 "° experience on any other 

transfer to the operation of this systeT rrr^^a^to^o^^^^^^^^^ 

Radar System was reported as: Actual equipmem Se^ d neln p '"^ °" '^^^ '26 

persons. Lecture and technical ordei-l persor "'"""^ P'"°"''' Equipment as installed onaircraft-4 

aircra^rsuS^^^^^^^^^^ 
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• , had experience on other radar systems. None were qualified to 
APQ 126 Radar System ana two had experience o 

troubleshoot the APQ 1 26 Radar System. i„,„,emented. was intended to teach: (1 ) 

...... The three ,eve,-tr.ni^^ operation, (3) the 

the purpose of the ,, 

performance of system self<hecks, and (4) the exercise ^^^^^ 

. u u .u« rix: pffpctivelv met the four training goals, iiity si""^ ' 
Each person was asked whether the ^PS e fee tive y m^^^^ questionnaire results and 

could perfoVm the objectives as a result °f I^V.^^'^^;^"^^^^^^^^ i^^ould be noted that the isolation o 
also the performance test scores on nialfunct on . j"" ,5 gcores on the performance test 

Sfuttio'ns is not possible without ^^'^^l-^XX^^^^^^^ '''t 
tr^S'lZ^y"^^^ iTrpSrarS^a^nronr felt that malfunction isolation training was not 

Table 1. Uaming on the GPS 




Combined 
(Qualified and Ungualined) 



Did you learn 
purpose of controls 
(Yes/Total) 

Can you interpret normal/ 
malfunction operation 
(Yes/Tot ai) 

Can you perform system 
self checks 
(Yes/Total) 

Can you isolate 

malfunctions 

(Yes/Total) 

Mean number 
replacements 
(Units) 

Median of average malfunction 
isolation time (Minutes) 



11/11 

11/11 

9/11 

10/11 

1.43 
1.76 



6/6 

6/6 

6/6 

5/6 

1.71 



17/17 
17/17 
15/17 
15/17 



qualified perwnnd ... '™™'f jj^'^o Kjo™ .'.»i"6- " 

General Purpose Simulator 5 persons 
Installed equipment -4 persons 
Actual equipment trainer - 2 persons 
Experienced technicians felt that the most effective mode of learning would be". 
Installed equipment - 8 persons 
Actual equipment trainer 3 persons 
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Mode of instruction making learning the easiest was felt by experienced technicians to be: 
General Purpose Simulator - 6 persons 
Installed equipment - 3 persons 
Actual equipment trainer - I person 

Attitudes and Acceptance. All personnel rjted the learning on the GPS as easy, while fourteen of 
seventeen Kked the GPS application to learmng. All but one of the experienced technicians recommended 
the use of a simulator such as the one evaluated for use m training the APQ 126 Radar System and other 
'Stems to be trained. Most frequently mentioned as suitable for generalized simulation were the A7 
cv^mputer and head up display (HUD) systems. Several also suggested the appropriateness for this approach 
in any or all avionics systems. 



IV. DISCUSSION 

The evaluation of the APQ 126 Radar System sinmlation on the GPS conducted at Buckley Field was 
designed to determine effectiveness, usability, and acceptance in a job environment. 

The goals which the simulator was designed to achieve were accomplished by all personnel. Not only 
were the students asked whether tiry had learned specific tasks or operations, a performance test also 
showed a high level of achievement using the GPS. Of partioular note is the programmed capability 
provided by the simulator cnabHng training in the isolation of selectee malfunctions. Experienced 
mechanics given the write-up averaged lesG than two minutes to identify and correct each malfunction in 
the simulation. Time to clear a malfunction on the fliglit Hne would average over an hour per malfunction 
because of the need to actually remove the faulty component, and probably slightly longer when training is 
also being accomplished. Thus, the time compression provided by the simulation enables more practice in 
developing the concepts involved in the isolation of malfunctions. The big picture is made clearer. 

All personnel exposed to the GPS indi^'ated that it provided an effective and efficient approach to 
learning. Learning was reported as easy on the GPS by everyone responding. Most reconMnended its use in 
Air National Guard training. 

Speed and ease of training made the GPS a preferred primary mode of instruction by experienced 
mechanics. The effectiveness when used in conjunction with actual equipment also was seen as indicating 
the usability of the GPS in training. With only minimal exposure to the GPS, many experienced mechanics 
selected it as a preferred primary training mode. The recommendaMons of these experienced mechanics 
indicate that the GPS is usable for field training. Further, the training program on the APQ 126 using the 
GPS was conducted by the least experienced of the weapons control shop mechanics-a three level person. 
He was able in only 214 hours apiece to provide satisfactory training in both normal and malfunction 
isolation mode for all students. Tlie GPS is shown as usable in field training by the present data. 

Student and instructor acceptance of the GPS was indicated by the questionnaire data. Experienced 
mechanics generally indicated that training could be performed faster and easier on the GPS. This would 
indicate acceptance. The GPS was recommended for training by most of the personnel. This also indicates 
acceptance. Wlien directly queried most personnel Hked using the APQ 126 Radar System simulation on the 
GPS. 

This report indic?tes the efficiency and effectiveness of the simulation of an APQ 126 Radar System 
in a field training environment. The 'csults are not unexpected. Simulation as a method of instruction can 
be effective for teaching man> tasks and skills in technical training. A wide variety of procedural sequences, 
perceptual motor skills, identifications, conceptual tasks and team functions have been effectively learned 
through the use of simulation. The approach of systematically abstracting and partially dupHcating tasks, 
activities, or operations can provide transfer of training from a synthetic environment to a real 
environment. Simulation allows student involvement in learning, paced to the needs of the individual. 
Learning by doing is emphasi/.ed. Both practice opportunities and forms of feedback usually not avjilable 
when using the actual equipment or when operating in the real world may also be provided. 
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Simulation often has training advantages over rcsl world operation. Baker and Warnick(l970) list six 
techniques to enhance transfer of training. Application of these techniques makes the training situation 
physically dissimilar to the actual situations. Tliese six conditions arc: 

1 . Provide augmented feedback, i.e., extra knowledge of results during instruction. 

2. Increase the number and frequency of crises, conflicts, equipment breakdowns, and emergencies. 

3. Reduce the operational time for certain everts, so as to increase the amount of practice. 

4. In the total perfonnance behavior, vary the part -task sequence, because jmall amounts of practice 
on several similar tasks promote more learning than large blocks of practice on a single task. 

5. Provide guidance and stimulus support in the early or initial stages of learning. 

6. During training, vary the progression of difficulty levels; a progressively easy-to-difficult 
procedure facilitates transfer of training. 

Simulators represent a real situation in which tasks are perfcnncd or operations are carried out. Tliey 
omit, however, selected parts of the real operation which are psychologically unimportant to the task of 
operation, which are dangerous, which are expensive, and hopefully they also omit some of the 
unpredictabilities of the real world. Simulators provide the learner with predetermined levels of control 
over the task or operation, thus allowing controlled practice on representative or critical aspects of the 
selected tasks or operations. 

For training cffectiv^i.^ss the simulator must provide psychological realism (Miller, 1954). Job inputs 
or inputs idcntifiably representative of job inputs must be provided. The student must exercise some l^vel 
of control over the system, typically based on the inputs. Some consequences as a result of the students 
interaction with the system also must be represented. Baker and War..ick (1970) state that operational 
similarity and motivational similaijy must be incorporated into the simulation. 

While simulation has been shown as an effective way of training for a wide variety of specific 
operations or tasks, the principles upon which the successes have been based remain unclear. In fact, our 
knowledge of the ps>cholog::al principle of transfer of training is stih incomplete. No predictive body of 
knowledge is available which will ensure the adequate design ot* a simulator for effective training. The 
specific goals of the training program, when used to direct the design of a simulator, generally result in a 
usable simulator, that is. a simulator which provide^ the required training. However, general factors which 
when manipulated, can reduce cost and*increase training effectiveness have not ben clearly identified. 

One major difficulty , widely recognized by training technologists concerning simulation is tl^.e issue of 
realism. Psychological realism does not imply reproducing all aspects of the physic.l erwironment. 
Engineering fidelity and physical realism are not necessarily incorporated into a GPS. A fixea sue face plate 
and a scries of selected images and meter readmgs hmit the possible visual inputs to the learner. Typically, 
other related units are not attached to the GPS although it has been donj when deemed necessary. 
Denenberg (1954) first showed that physical realism in a tank hull trainer may not be necessary in 
providing necessary and adequate transfer to the job. While no generalizablc simulation studies have been 
reported, results similar to Denenberg*s study have been reported for a wide variety of Simulators. In fact, 
AF Pamphlet 50-58 (Vol IV, Section 5-lOf) provides specific guidance in selecting appropriate levels of 
representation. This section states that o:.ly those properties relevant to the learning task should be 
represented. An example is provided showing that either too much physical realism or too little realism are 
inappropriate in selecting the appropriate .eprerentation levels. The visual materials used and selected must 
provide the necessary inputs to enable the cuing of appropriate task performances. Physical realism provides 
no assurance that useful information will be seen, learned, or remembered. Forexampie physical reality 
may be too complex for a beginning learner to make the appropriate discriminations, associations, and 
generalizations so as to meet the training requirements. Representations of reality frequently must be 
simplified and stylized in the early stages when learning efficiency is desired (Travers, 1964). 

It has been customary in procuring new simulators to require engineering fidelity; that is, the 
simulator is required t j function as nearly as possible in the way in which the real equipment functions. 
Fnis usually means ihat an AET will be preferred. The implicit assumption is that better transfer can be 
directly associated to more realistic representations. Not only may this assumption be expt .sive, it may 
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APPENDIX A: PRETESTING AND POSTTESTING QUESTIONNAIRE 
PLEASE ANSWER THESE QUESTIONS BEFORE TRAINING 

SSN RANK/GRADE YEARS OF SERVICE 

Primary AFSC 
Secondary AFSC 

Have you had any experience with the A-7 aircraft? Yes No 
I have experience working on equipment on the following aircraft 
series: 



Are you familiar with the APO 126 Radar System? 


Yes 


No 




Have you received training on the APQ 126 
Radar System? 


Yes 


No 




If yes, did you learn from: 








(a) Actual equipment trainer 


Yes 


No 


P 


(b) Equipment as installed on the A-7 aircraft 


Yes 


No 


P 


(c) TO and lecture 


Yes 


No 


P 


(d) EC II Simulator 


Yes 


No 


P 


(Please circle P for primary mode of training) 








Have you worked on the APQ 126 Radar System? 


Yes 


No 





Are you qualified to troubleshoot the APQ 126 

Radar System? Yes No 

Have you worked on other radar systems? Yes No 

If yes, which 
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PLEASE ANSWER THESE QUESTIONS FOLLOWING TRAINING 



SSN 



1. Did you like using this approach to learning? Yes Nc 

2. Was the learning (a) easy or (b) difficult when using the ECU 
Simulator? 

(circle one) (a) (b) 



3* 


Did you learn the purpose of the controls? 


Yes 


No 


4. 


Can you interpret normal versus abnormal 
(malfunction) operation? 


Yes 


No 


5. 


Can you perform the appropriate system self checks? 


Yes 


No 


6* 


Can you isalate malfunctions? 


Yes 


No 


?• 


Will this training enable you to work more 
effectively on the actual equipment? 


Yes 


No 



8, As a training person using as a primary mode of 
instruction each of the following modes 

(a) Actual equipment trainer 

(b) Equipment as installed on A/C 

(c) TO and lecture 

(d) EC II Simulation 

Which mode would be the fastest? (a) (b) (c) (d) (Rank order 

with top 

Which mode would provide most choice //I) 

effective learning? (a) (b) (c) (d) 

Which mode would make learning 

the easiest? (a) (b) (c) (d) 

9. Would you recommend the use of a simulator such as the EC II 
simulation of the APQ 126 for your training program? 

(a) For the APQ 126 Yes No 

(b) For other systems to be trained Yes No 
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